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Introduction 



Mediation has become both an effective means to address concerns related to special education and early 
intervention services throughout the country and an alternative to contentious and litigious methods of dispute 
resolution. The process of mediation holds great promise for assisting parents, children, advocates, education 
agencies, and others in developing appropriate solutions to a wide range of concerns. 

Evaluating a mediation program is important and helps ensure that services are continually improved and refined. 
The effort devoted to evaluation and program improvement enhances the likelihood that mediation programs 
and practices will be effective and utilized to the greatest degree possible. 

This resource guide presents an overview of mediation evaluation with a specific focus on how to make use of 
participant questionnaires in evaluating mediation programs. This guide seeks to achieve the following purposes: 

• To help readers understand evaluation processes and outcomes 

• To provide instruments for evaluating the effectiveness of mediation services 

• To help program mani^ers and others use evaluation data to identify appropriate strategies for 
improving mediation practice 

While designed to help program managers and others understand evaluation processes and outcomes, 
this guide is not intended to be a comprehensive how-to manual on evaluation. Evaluation of social services is 
a highly specialized endeavor, and program managers should, if possible, consult with a program evaluation 
expert for guidance and assistance. Often assistance is available through a local university or nonprofit 
mani^ement organization. For some types of program evaluation, an external evaluator is needed to ensure 
the credibility and objectivity of the findings. This resource guide is more narrowly focused on evaluation 
procedures using participant feedback. The approach presented in this resource guide may easily be used by 
a program manager or other staff for ongoing assessment of program performance and improvement planning. 
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Section 1 . Establishing the evaluation context 



Special education mediation occurs within a broad framework of participant expectations, professional 
mediation practices, and legal guidelines and rules. Federal, state, and administrative laws and regulations as 
well as political, professional, and social realities set the context for evaluation. Among the organizations and 
people involved are state and local education agencies, mediators, parents and parent advocates, and other 
concerned individuals and organizations representing the cultural and ethnic diversity of the local school system 
and its constituents. When a program manager estabUshes an evaluation process for a mediation program, 
many or all of the above players can be important stakeholders and valuable aUies in that process. 

These stakeholders need information that will enable them to make decisions regarding the effectiveness of 
the mediation program and potential areas for improvement. While program evaluation could include a broad 
range of methodologies, this guide focuses on the use of questionnaires to collect feedback from mediation 
participants. Parents, school personnel, and mediators may each have differing and vahd perspectives on how 
well a particular mediation worked and how successful its outcomes were. Capturing these perceptions can 
provide a mediation program manager with valuable insights into how to increase the success of mediation services. 



The table below outlines the steps in designing and conducting an evaluation using this stakeholder 
approach and focusing on participant feedback. 



ACTION STEPS IN THE DESIGN & IMPLEMENTATION OF AN EVALUATION PLAN 


1. 


Identify stakeholders and key individuals related to mediation services and evaluation. 

This group includes: state and local education agency personnel, parent advocates,“students 
with disabilities, cultural/ethnic representatives, parents, mediators, and evaluators. 


2. 


Define the role of stakeholders and key individuals for the evaluation process and clarify 
decision-making procedures. 


3. 


Establish the purpose of the evaluation (e.g., system improvement, training needs, 
customer satisfaction, etc.) 


4. 


Identify the key performance indicators and measures that will be useful to stakeholders 
as they advise the evaluation process. 


5. 


Determine and secure the resources-including evaluation materials, staff members, 
and other supports-that are needed to carry out the evaluation. 


6. 


Use the appropriate instruments and methodologies to conduct the evaluation. 
These might include questionnaires, interviews, observations, or focus groups. 


7. 


Collect and analyze the data. 


8. 


Use the data to identify potential Improvement strategies and opportunities. 


9. 


Implement the appropriate strategies. 


10. 


Conduct an evaluation to measure the impact of the improvement strategies. 
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Section 2. Considerations in the evaluation process 

Before turning to the specifics of survey or questionnaire items and improvement strategies, a number of 
important issues need to be considered: the use of questionnaires for evaluating mediation; the accessibUity of 
data collection formats for inclusion of diverse participants; the maintenance of confidentiahty for participants; 
the identification of key performance measures of mediation; and guidelines for data analysis. 



Use off questionnaires ffor evaluating mediation 



While other evaluation methods could be used to assess participant perspectives on mediation, 
questionnaire-based survey evaluation suits mediation programs because questionnaires 

• Allow for collection of information on a wide range of topics, 

• Require a relatively short time for respondents to complete, 

• May be completed at times convenient to the respondent, 

• Do not require extensive training of staff or mediators to administer, 

• Allow for translation into alternative formats and languages, 

• Provide data that can be easily collected and analyzed, and 

• Provide findings that may be summarized and presented clearly. 



Questionnaires may be distributed in person, by mail, or online. Each approach has its benefits and drawbacks. 
While it may be cost-effective to ask participants to complete a questionnaire before leaving the mediation 
session, it may lead to responses that are influenced by the short-term emotional state of the participants — 
sometimes feelings of accomplishment, other times feelings of exhaustion. Another potential drawback stems 
from the choice of who distributes the questionnaires; if the mediator hands them out at the conclusion of a 
session, participants may be concerned that the mediator will see their responses and therefore may be reluctant 
to comment candidly about their experience. This may be addressed by having the mediator provide an 
envelope along with the survey, with instructions on how to return it directly to the mediation program manager. 

Questionnaires distributed by mail avoid the complications of those distributed in person, but may be more 
costly due to maiUng expenses. Another potential drawback of mailed questionnaires — especially follow-up 
questionnaires administered a few months after the mediation — is a low response rate. Including a 
pre-stamped, pre-addressed envelope in which to return the completed questionnaire and a cover letter 
emphasizing the importance of the evaluation process will help to increase the response rate, but, even with this 
letter, mailed questionnaires seldom receive the response rate that those distributed in person receive. 

However, a reminder phone call or postcard may stimulate respondents who have not returned their surveys. 



The questionnaires provided in this resource guide are intended to serve as models or templates for program 
managers to adapt to their specific program features. Whenever possible, program managers should survey the 
participants in all mediated cases. Random sampling may be useful for programs with very large caseloads, 
but as questionnaires are relatively inexpensive, most programs are advised to survey all cases to gather as 
much data as possible. 



ERIC 



Using Participant Feedback to Evaluate and Improve Quality in Mediation 



9 



3 



Online questionnaires have many advantages including the savings related to mailing costs and the time required 
to process and compile the responses. Once a mediation provider or education agency has a page on the 
World Wide Web, the development of an online questionnaire is not difficult. The disadvantage of relying solely 
on online resources, of course, is that many families do not have easy access to the Internet. 



Accessibility of data collection formats for inclusion of diverse participants 

Data collection should be accessible for every mediation participant, so particular care needs to be taken to 
ensure that materials and methods are available in a variety of formats. To ensure that evaluation is conducted 
appropriately and respectfully, questionnaires should be reviewed for accessibility for all possible respondents 
and users of these materials. For some participants, interviews may be a preferable approach. 

To ensure broad participation, questionnaires should be available in large-print, Braille, and in languages other 
than English. Interviews should be conducted at accessible facilities and interviewers prepared to accommodate 
a wide range of languages and language skills as well as cultural contexts. Regardless of the data collection 
method, sufficient time should be provided to allow the participants to complete the evaluation. 

More information about culturally appropriate practices can be found in Keys to Access: Encouraging the 
Use of Mediation by Families from Diverse Backgrounds (Engiles, Fromme, LeResche, & Moses, 1999)- 
This document offers practical strategies for collaborating with families and school system personnel that 
have different cultural, linguistic and socio-economic backgrounds. 



Maintenance of confidentiality for participants 

To ensure that participants provide open and honest responses to evaluation instruments, it is incumbent on 
program managers to safeguard the anonymity of the individuals and the confidentiality of their responses. 

In the context of special education and early intervention services, this responsibility is heightened by federal 
and state confidentiality provisions and by any additional confidentiality agreements that are entered into by 
the participants. 

Participants will expect that any information used for evaluation purposes will be treated as private and 
confidential. Assurances of confidentiality should appear on every written questionnaire and be conveyed as part 
of every structured interview or observation. Prior to undertaking any evaluation, agencies must determine which 
data to collect and how that data will be treated. For example, is any of the material open to public scrutiny? 

If the evaluation process calls for follow-up contacts with participants, how will an identifying code be developed, 
and where will the list that links the code to specific cases be kept? All policies related to the data collection 
effort should be made available to participants at every stage of the evaluation. 
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Identification of key performance measures of mediation 



The evaluation of mediation involves multiple stakeholders and multiple informants. The program manager 
is responsible for ensuring that evaluation takes place and that the findings are used to improve service. 

School personnel, family members, and mediators are both stakeholders and informants — they provide 
information and, to differing degrees, use evaluation information to make decisions about fiiture courses of action. 
The performance issues that are critical to the varying purposes of mediation evaluation are listed below: 



PERFORMANCE AREAS 


KEY PERFORMANCE ISSUES 


Program Efficiency 


Cost to participants 

Time from referral to resolution 


Program Effectiveness 


Outcomes of mediation 

Participant satisfaction with mediated outcomes 
Durability of mediated outcomes 
Impact on relationship between participants 
Program neutrality 


Mediation Process 


Appropriateness/Usefiilness 
Preparation process and materials 
Fairness (opportunity to tell story, feeling understood, 
respectful treatment, control over outcomes) 


Mediator Performance 


Skills of the mediator 
Knowledge of the mediator 
Impartiality of the mediator 



Guidelines for survey data analysis 

Most of the information collected by mediation surveys is attitudinal — that is, the survey instruments are 
designed to gauge the perspective or feelings of the respondent. A common form of attitudinal survey is the 
ordinal rating scale, which presents responses in some order like bad-to-good or dissatisfied-to-satisfied. 
Social scientists usually order the responses from negative to positive as a means to ensure that the respondent 
is reading through the various degrees of opinion, instead of simply checking the first box. 
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To facilitate accurate data gathering, surveys should have code numbers next to each response. 
Without these numbers, recording survey results is quite tedious and prone to error. 

In the sample questionnaires presented in this guide, note that every response has such a code: 

□ 1 Very dissatisfied 

□ 2 Somewhat dissatisfied 

□ 3 Somewhat satisfied 

□ 4 Very satisfied 



Note that the numbers themselves are only placeholders and don’t have a numeric value in relation to each other. 
Thus, a “4” is not twice as good as a “2”, as demonstrated in the examples below. 

There are many ways to analyze the data from rating scales, some of which are complex and require 
sophisticated statistical analysis. But as this guide is primarily concerned with identifying opportunities for 
improvement, extensive statistical analysis will typically not be justified. A simple analysis of tabulated responses 
will reveal those program areas that require attention. Consider the following two response sets, based on a 
survey of thirty-four mediation participants. 

Number of responses 



Question 1 


1 


1 1 1 Very dissatisfied 




3 


1 1 2 Somewhat dissatisfied 




9 


1 1 3 Somewhat satisfied 




21 


1 1 4 Very satisfied 


Question 2 


4 


1 1 1 Very dissatisfied 




15 


1 1 2 Somewhat dissatisfied 




10 


1 1 3 Somewhat satisfied 




5 


1 1 4 Very satisfied 



For Question 1, the responses are mostly positive, with 30 respondents answering either “Somewhat satisfied” 
or “Very satisfied,” and only 4 answering “Somewhat dissatisfied” or “Very dissatisfied.” Contrast that distribution 
with the responses to Question 2, which yielded 15 responses of “Somewhat satisfied” or “Very satisfied,” 
but 19 responses of “Somewhat dissatisfied” or “Very dissatisfied.” The high proportion of negative responses 
demonstrates that the program area addressed in Question 2 needs attention and improvement. 

There is one additional factor to consider: the sample size. If an evaluation project has only a handful of 
completed questionnaires to analyze, the themes or trends in the data reflect only a few participants’ attitudes, 
and these may not be representative of the typical participant. As a general guideline, evaluators should strive 
to get as many surveys completed as possible. 

12 
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Section 3. A structured approach to evaluating mediation 



This section is organized around the performance measures identified in the table in Section 2 on page 5. 

Each performance measure is addressed individually in a three-part structure; 

• Definition and discussion of the measure. 

• A sample question related to the measure. The question is drawn from the survey instruments 
presented in this resource guide. 

• Strategies for improving each performance area. These suggestions have been developed in 
consultation with a number of program managers, parents, educators, and other mediation 
authorities to serve as a starting point for evaluation teams and program managers. Each of these 
suggestions is useful only to the degree that it responds to a documented need and to the context in 
which that need exists. Therefore, for any given special education mediation program, some of these 
suggestions may be very appropriate while others may be less effective. The unique features and 
context of a program will necessitate additional strategies. 



Program efficiency: Cost to participants 

In many instances, especially when assessing new programs, evaluation is used to determine the cost efficiency 
of dispute resolution to participants. As McDougall (1998) noted in “Evaluating the Fiscal Effectiveness of 
ADR Processes in Special Education Settings:” “Given the high cost and high stakes decision making involved 
in disagreements in the area of special education, it is appropriate to turn attention to the discussion of 
developing a fiscal capability model that demonstrates effective use of ADR procedures in both tangible and 
intangible returns on monies and resources invested by school districts, parents and others.” 

Sample question 

Q. Compared to other hkely means of resolving this matter, how did mediation affect the cost of this case to 
you or your agency? Please check one; 



□ I Mediation cost more 

□ 2 No change in cost 

□ 3 Mediation cost less 
I 1 4 Don’t know 
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Suggestions to improve cost to participants 

• Develop a mediator roster with broad geographic availability to reduce travel costs 

• Establish guideUnes for length and/or number of mediation sessions 

• Triage cases to determine whether mediation is appropriate 

• Explore conciliation options prior to full mediation 

• Conduct pre-mediation meetings or conference calls to ensure adequate preparation by all participants 

• Conduct intake (and possibly mediation) online 

• Make mediation option available prior to due process fibng 



Program efficiency: Time from referral to resolution 

Dispute resolution processes such as mediation often take less time to schedule, prepare for, and deliver than 
judicial or administrative processes. This efficiency is promoted widely. The time required to process a case is 
a common measure of program evaluation often labeled “turnaround time” or “time to disposition.” While this 
may appear to be a straightforward measure, consider the possible meanings. Does it represent the time 
between fibng for a due process hearing and convening a mediation? The time between referral to the mediation 
provider and the deUvery of a written agreement or other report? Any of these measures might be useful, as long 
as the same measure is used throughout an evaluation. 

One common measure of the time efficiency of mediation is the time between referral and the conclusion of the 
dispute resolution process. This is a useful record to maintain, particularly when comparative data exits for 
other dispute resolution processes. 

Sample question 

Q. Compared to other Ukely means of resolving this matter, how did mediation affect the time spent addressing 
this matter? Please check one: 



□ 1 Mediation increased the time spent 

□ 2 No change in time spent 

□ 3 Mediation shortened the time spent 

□ 4 Don’t know 



Suggestions to improve time from referral to resolution 

• EstabUsh guideUnes for the length of mediation sessions 

• Develop a broader roster of mediators to gain scheduUng flexibffity 

• Conduct mediator training to enhance skiUs in developing a focused statement of problems and 
identifying manageable issues that may be resolved through mediation 

• Minimize the time between a request for mediation and the deUvery of services through automated 
case management (preparation of letters and forms), maiUng of calendars for participants to select 
available times in the near future, or onUne scheduUng 

• Emphasize the use of single sessions instead of multiple sessions 1 4 
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• Conduct pre-mediation meetings or conference calls to ensure adequate preparation by all participants 

• Schedule the due process hearing within a few days of the mediation in case full agreement is not reached 



Program effectiveness: Outcomes of mediation 

Like the measure of time efficiency, one may presume that measuring the outcome of a dispute resolution process 
is simple: cases conclude either in an agreement or they do not. However, this breakdown may not account for 
many cases. Consider how the following cases would fit into this system of “agreement” or “no agreement”: 

• Cases in which participants “concihate” — that is, reach an agreement after the time of referral 
(or intake) to mediation but before the actual mediation session. 

• Cases that reach agreement after the mediation session ends, based on the relationships that have 
developed and the information that was exchanged in the mediation. 

• Cases in which agreements are reached on some issues, but not all issues. 

Sometimes mediation programs will report an “agreement rate,” which is the number of cases settled divided 
by the number of cases mediated. But what are we to make of this figure? A low rate of agreements may 
indicate that the program is not outcome-effective or cost-effective. An unusually high rate of agreements may 
indicate that neutrals are employing undue pressure or may be imposing settlements on the participants. 

Program evaluators might consider augmenting their quantitative data with qualitative data to achieve a more 
accurate picture of how participants perceive their experience with the mediation service. 

Sample question 

Q. What was the outcome of the mediation? Please check one: 

□ 1 No agreements 

□ 2 Agreement on some but not all issues 

□ 3 Agreement on all issues 

Suggestions to improve mediated outcomes 

• Conduct mediator training on drafting effective agreements, including provisions for monitoring 
implementation and comphance 

• Consider the use of longer and/or multiple mediation sessions to attend to all disputed issues 

• Allow participants to select their own mediator to encourage rapport and trust 

• Enhance participant preparation by providing information on the process and the relevant laws and 
guidehnes, and an overview of mediation and negotiation practices 

• Clarify the issues under dispute prior to the mediation session 

• Confirm that all key decision-makers are present or available to develop appropriate agreements 
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Program effectiveness: Satisfaction with mediated outcomes 

Just as the mediated outcome is an important indication of a program’s effectiveness, so is the participants’ 
satisfaction with the outcome. The distinction between the outcome (agreement reached on all issues, 
agreement reached on some but not all issues, no agreements reached) and the participants’ satisfaction may 
seem subde, but consider that while some participants may be dissatisfied about not reaching an agreement, 
others may find that outcome acceptable. On the other hand, some participants may reach an agreement in 
order to avoid a less desirable result, such as a due process hearing or the continuation of an undesirable 
program or placement, and therefore not be altogether satisfied with that agreement. 

Sample question 

Q. How satisfied are you with the outcome of the mediation? Please check one: 

□ I Very dissatisfied 

□ 2 Somewhat dissatisfied 

□ 3 Somewhat satisfied 

□ 4 Very satisfied 



Suggestions to improve participants’ satisfaction with mediated outcomes 

• Consider use of longer and/or multiple mediation sessions to attend to all issues under dispute 

• Allow participants to select a mediator by mutual agreement in order to encourage rapport and trust 

• Enhance participant preparation by providing information on the process and the relevant laws and 
guidehnes, and an overview of mediation and negotiation practices 

• Confirm that all key decision-makers are present or available to develop appropriate agreements 

• Provide opportunities early in the mediation for clarification of expectations, issues, and 
suggested remedies 

• Conduct reality-checks during the mediation 



Program effectiveness: Durability of mediated outcomes 

In many mediation contexts, the durability of an outcome is easily measured. For example, the setdement terms 
of a civil claim often involve only the exchange of money or services. Once this exchange has taken place, the 
setdement has been fulfilled. This is labeled the “compfiance rate,” which is often measured by a two-point scale: 
agreement followed, or agreement not followed. Compfiance may not be quite as straightforward in special 
education mediadon where agreements can be complex and sometimes are adjusted following the mediation 
based on intervening factors. 

Obviously, durability cannot be measured immediately following a mediadon session. To assess how well agreements 
were upheld, a follow-up questionnaire or interview must be conducted. Many mediation programs conduct these 
follow-ups after three months have passed, while others choose to wait six months or even a year. While the specific 
amount of time between mediadon and follow-up is not critical, it is important to wait a uniform length of time to 
ensure consistent data across all cases. 
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Sample questions (for a follow-up survey) 

jQ. If an agreement was reached in mediation, have the provisions of the agreement been followed or implemented? 

I I I None of the provisions have been followed or implemented 
I I 2 Some (but not all) provisions have been followed 
or implemented 

I I 3 All provisions have been followed or implemented 



jQ. If an agreement was not reached in mediation, how was the matter resolved? 

I I I Through a due process hearing 

I I 2 Through a complaint fihng 

I I 3 Through an lEP meeting 

I I 4 Through an informal agreement or settlement 

I I 5 The concern was dropped 

I I 6 The concern remains outstanding 



Suggestions to improve the durability of mediated outcomes 

• Conduct mediator training on feasibihty/reality testing 

• Develop a system to monitor agreement adherence 

• Conduct mediator training on drafting effective agreements, including contingency provisions as appropriate 

• Ensure that decision-making authority is present or available 

• Gain a commitment to incorporate mediated agreements into the lEP (as appropriate) 

• Establish a commitment with the participants to re-mediate any implementation problems or 
concerns and incorporate into the written agreement 



Program effectiveness: Impact on relationship between participants 

Improving the relationship between participants in early intervention and special education disputes is among 
the greatest promises of cooperative processes such as mediation. When participants develop and maintain a 
constructive relationship, it paves the way for a more collaborative partnership in the future and lessens the 
likelihood of future complaints arising. If problems do arise, participants can be expected to communicate 
and negotiate at an earher point and with less distrust and antagonism. 

Sample question 

Q. When you compare your situation before and after the mediation, how has mediation affected the 
relationship between the family and the school? Please check one: 

□ I Mediation has harmed the relationship 

□ 2 Mediation has had httle or no effect on the relationship 
I I 3 Mediation has improved the relationship 
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Suggestions to enhance mediation’s impact on the relationship between participants 

• Develop guidelines for participation in mediation that encourage constructive behavior 

• Conduct mediator training to emphasize opportunities for direct contact and relationship-building 
between participants 

• Determine an appropriate mediation approach based on cultural or other expectations 

• Conduct mediation in a comfortable, neutral site 

• Conduct mediation with greater informality and increased sensitivity to the cultural needs of participants 

• Provide food or beverages during mediation 

• Focus the mediation on future openness and collaboration among participants 

• Emphasize the shared interest among educators and family members in the child’s education 

• Encourage mediators to include provisions for future communication between participants in 
mediation agreements 



Program effectiveness: Program neutrality 

The appearance and reality of neutrality in the program is another important consideration. For example, when 
special education mediation services are coordinated through a state or local education agency, families and 
their advocates may have concerns that the mediation program will be biased in favor of the education agency. 

It is valuable to ask participants how they assess the neutrality of both the mediator and the agency coordinating 
the mediation services. The impartiality of the mediator will be addressed later in this document. 

Sample question 

Q. Concerning the impartiality of the special education mediation program (not the mediator), how do you feel? 
Please check one: 



□ 1 The mediation program favors schools 

□ 2 The mediation program favors famiUes 

□ 3 The mediation program is neutral, and favors neither party 



Suggestions to enhance program neutrality 

• Utihze a state advisory board that is representative of the program stakeholders, including famihes, 
educators, and mediators 

• Consider contracting out the mediation services to an independent agency, perhaps a community 
mediation program or a pubUc or private dispute resolution provider 

• Clarify the role and administrative location of the mediation program 

• Ensure a mediator roster or pool that reflects the communities served — consider the age, gender, 
ethnicity, socioeconomic, and other traits of the mediators 

• Hold mediation sessions at neutral locations accessible and acceptable to all participants 
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Mediation process: Appropriateness and usefulness 



The participants’ perception of how appropriate and useful mediation was to their situation may affect 
the number of parents and administrators who will choose to use mediation in the future. Instead of asking, 
"How appropriate did you find mediation for addressing this caseF", a questionnaire may frame the 
question in operative terms: “Would you choose to use mediation in a similar situation in the future?” 
Another question to measure satisfaction with the usefulness of mediation might be, “Would you recommend 
mediation to a friend in a similar situation?” 

Sample questions 

Q. How productive or unproductive was mediation for this matter? Please check one: 

□ 1 Mediation was very unproductive 

□ 2 Mediation was somewhat unproductive 

□ 3 Mediation was somewhat productive 

□ 4 Mediation was very productive 

Q. Would you recommend mediation to another family or school in a similar dispute? 

I I 1 No 
I I 2 Yes 



Suggestions to improve appropriateness and usefulness 

• Provide an orientation session to prospective participants 

• Develop intake procedures with case assessment and screening protocols that include information 
about previous settlement attempts 

• Determine an appropriate mediation approach based on cultural or other expectations 

• Prior to mediation, confirm that participants understand and desire to proceed with mediation 

• Explain the benefits and Umitations of mediation in education and outreach efforts 

• Develop an “agreement to mediate” form to ensure the participant's acceptance of the mediation 
process and the other participants 



Mediation process: Preparation process and materials 



For many participants in early intervention and special education cases, mediation is a new and unfamiliar 
process. Consequently, proper orientation and advance preparation by participants can be an important part of 
the mediation process. 



As well as an in-person consultation, many programs also make available written materials or videotapes that 
outhne the mediation process and show how to prepare and participate effectively. Evaluation questions may be 
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framed to ask about either the overall preparation process or individual parts of the process. For example, 
“How satisfied were you with the videotape you viewed prior to the scheduled mediation?” 

Sample question 

Q. Based on the information, meetings, or materials provided by the mediation program, how prepared 
did you feel for the mediation? Please check one: 

□ 1 Very unprepared 

□ 2 Somewhat unprepared 

□ 3 Somewhat prepared 

□ 4 Very prepared 



Suggestions to improve the preparation process and materials 

• Provide a toll-free phone number for the mediation coordination office 

• Provide an orientation session to prospective participants 

• Provide materials as far in advance of the scheduled session as possible 

• Make materials available by as many methods as possible, including mail, email, fax, and o nlin e 

• Provide the same preparation materials to all participants 

• Ensure that materials are accessible for all participants, including individuals who may require 
alternate formats and languages 

• Develop an orientation videotape to show to participants prior to mediation 

• Provide opportunities for prospective participants to speak with past participants 

• Elicit ideas and concerns from participants related to the preparation process 



Mediation process: Fairness of the process 

Perhaps the most ambiguous term commonly used in mediation is “fair.” What constitutes “fairness” and 
how can it be measured? Due to this ambiguity, many researchers have moved away from asking participants 
whether the process was fair, and have developed questions related to “procedural justice.” Measures of this 
might include the following descriptors of participant experience: 

• sufficient opportunity to express her/his point of view 

• a sense of being understood by the mediator 

• respectful treatment by the mediator 

• an appropriate level of control over any decisions made 

To ascertain whether all of these components of procedural justice have been addressed, evaluations may 
contain questions related to each of them. (As impartiality is critically important, it will be addressed separately 
as a factor of mediator performance.) In an effort to break down fairness into its constituent parts, participant 
surveys often employ some of the following questions. 

20 
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Sample questions 

Q. To what extent did you have sufficient opportunity to relate your issues and concerns during the mediation? 
Please check one: 



□ 

□ 

□ 



1 Had no opportunity to relate issues and concerns 

2 Had some opportunity to relate issues and concerns 

3 Had full opportunity to relate issues and concerns 



Q. How well did the mediator understand your issues and concerns? Please check one: 

□ I Did not understand at all 

□ 2 Understood partially 

□ 3 Understood fully 



Q. How respectful or disrespectful was the mediator toward you? Please check one: 

□ I Very disrespectful 

□ 2 Somewhat disrespectful 

□ 3 Somewhat respectful 

□ 4 Very respectful 



Q. If you reached any agreements, do you feel you had an appropriate level of input in determining the 
content and terms of those agreements? Please check one: 

□ ■No 

□ 2 Yes 



Q. Did you feel pressured by the mediator to reach an agreement? Please check one: 

□ I Yes 

□ 2 No 



Suggestions to improve fairness 

• Develop a protocol for speaking order; e.g., the complainant or parent speaks first 

• Provide mediator training on strategies for ensuring balance in participants’ roles 

• Provide mediator training with a focus on non-evaluative approaches 

• Encourage participants to bring support persons or advocates 

• Allow sufficient time for participants to relate concerns fully 
O • Provide participant feedback directly to the mediator 
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Mediator performance: Skills of the mediator 



Mediation skills include listening, questioning, problem-solving and, in some models, offering ideas that may 
lead to resolution. Each of these skill areas may be measured independently. 

Sample question 

Q. How skilled was the mediator at listening and understanding your concerns? Please check one: 

I I I Not at all skilled 
I I 2 Somewhat skilled 
I I 3 Very skilled 



Q. How skilled was the mediator at asking relevant questions? Please check one: 

I I I Not at all skilled 
I I 2 Somewhat skilled 
I I 3 Very skilled 



Q. How skilled was the mediator in helping you to generate and consider options for resolving the dispute? 
Please check one: 



I I I Not at all skilled 
I I 2 Somewhat skilled 
I I 3 Very skilled 



Suggestions to improve the skills of the mediator 

• Limit the roster of mediators to those with a strong record of service 

• Employ quality screening or a certification process for mediator admission onto the roster 

• Collaborate with local mediation and education resources for training, including universities 
and professional schools 

• Provide mediator training with a skill evaluation component 

• Develop a mentoring system among mediators 

• Conduct regular mediator staff meetings 



Mediator performance: Knowledge of the mediator 



O 
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Mediators should be knowledgeable about the laws and regulations relating to the provision of special education 
and related services. Some programs may also want mediators to have additional knowledge regarding specific 
disabilities, particular educational programs or services. This question may provide useful information to 
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program administrators if multiple surveys indicate a concern about a specific mediator’s level of knowledge. 
Caution should always be exercised in drawing conclusions about a mediator based on a single survey. 

Sample Question 

Q. How knowledgeable of relevant laws and regulations was the mediator? Please check one: 

□ I Not at all knowledgeable 

□ 2 Somewhat knowledgeable 

□ 3 Very knowledgeable 



Suggestions to improve the knowledge of the mediator 

• Conduct a needs survey of mediators regarding knowledge areas 

• Provide mediator training on related laws, regulations, and guidelines 

• Develop a testing process to ensure mediator familiarity with relevant topics 

• Create a system for regular updates or refresher seminars for mediators 

• Make resource people available during the mediation 

• Conduct peer-review or critique of mediation agreements 



Mediator performance: Impartiality of the mediator 

Impartial, unbiased and balanced treatment of each mediation participant is an essential characteristic of mediation. 
While a range of mediator styles has emerged in recent years, the perception of mediator neutrahty remains critical 
to the acceptance of the mediation process by its participants. Note that this measure is specifically related to the 
mediator, while program neutrahty is measured separately. 

Sample question 

Q. Concerning the impartiahty of the mediator, how did you feel? Please check one: 

□ 1 The mediator favored the school 

□ 2 The mediator favored the family 

□ 3 The mediator was neutral, and favored neither party 



Suggestions to improve the impartiality of the mediator 

• Utihze mediators who have no present or past association with the participants (educators or 
parents as mediators may be viewed as partial toward one party) 

• Provide mediator training with an emphasis on impartiahty and neutrahty 

• Ensure mediator disclosure of relahonships and affihations 

• Provide opportunihes for participants to select their mediators through mutual agreement 
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Conclusion 



This resource guide provides a model for using participant feedback to improve mediation practice and 
programs. The use of participant questionnaires can provide an important source of information to evaluation 
teams seeking to utUize an ongoing feedback loop, where program improvements are continually evaluated and 
refined. The reader should note that the materials and methods presented here are to be used within the larger 
evaluation context of stakeholders described in Section 1. Representatives from interested parties (i.e., parents, 
school personnel, mediators, program managers) should participate in the design, collection and analysis of 
evaluation data and in the identification of improvement strategies. 

Mediation affords families and schools a valuable opportunity to resolve disagreements and preserve 
relationships among the parties involved. The effort devoted to evaluation and program improvement increases 
the UkeUhood that mediation programs and practices will be effective. Ultimately, mediation can enhance the 
partnership between families and educators that is critical to the development of quality educational program 
for students. 
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Sample exit survey — The following survey may be administered as a written questionnaire or an interview. 

The [SEMS agenq^] conducts ongoing evaluation of its services. Your input is essential to that evaluation. 

Please take a few minutes to respond to the following questionnaire about your experience in mediation. 

Your responses are considered confidential and any reproduction of your comments will contain no reference 
to your identity. 



Please identify your role in the mediation: 

□ 1 Family member 

□ 2 School representative 

□ 3 Family advocate or attorney 

□ 4 School advocate or attorney 



The following questions relate to the 
1 . What was the outcome of the mediation? 



mediation session. 

Please check one: 

□ 1 No agreements 

□ 2 Agreement on some but not all issues 

□ 3 Agreement on all issues 



2 . Based on the information, meetings, or materials provided by the mediation program, how prepared 
did you feel for the mediation? Please check one: 

□ 1 Very unprepared 

□ 2 Somewhat unprepared 

□ 3 Somewhat prepared 

□ 4 Very prepared 



3 . To what extent did you have an opportunity to relate your issues and concerns during the mediation? 
Please check one: 



□ I Had no opportunity to relate issues and concerns 

□ 2 Had some opportunity to relate issues and concerns 

□ 3 Had full opportunity to relate issues and concerns 
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4. If you reached any agreements, do you feel you had an appropriate level of input in determining the content 
and terms of those agreements? Please check one: 



□ .No 
I I 2 Yes 



5. When you compare your situation before and after the mediation, how has mediation affected the relationship 
between the family and the school? Please check one: 

□ 1 Mediation has harmed the relationship 

□ 2 Mediation has had little or no effect on the relationship 

□ 3 Mediation has improved the relationship 

6. How productive or unproductive was mediation for this matter? Please check one: 

□ 1 Mediation was very unproductive 

□ 2 Mediation was somewhat unproductive 

□ 3 Mediation was somewhat productive 

□ 4 Mediation was very productive 



The following questions focus on the role of the mediator. 

7. How well did the mediator understand your issues and concerns? Please check one: 

□ 1 Did not understand at all 

□ 2 Understood partially 

□ 3 Understood fully 



8. How respectful or disrespectful was the mediator toward you? Please check one: 

□ 1 Very disrespectful 

□ 2 Somewhat disrespectful 

□ 3 Somewhat respectful 

□ 4 Very respectful 

9. Did you feel pressured by the mediator to reach an agreement? Please check one: 

□ -Yes 2Q 

□ 2 No 
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10 . How skilled was the mediator at listening and understanding your concerns? Please check one; 



□ I Not at all skilled 

□ 2 Somewhat skilled 
I I 3 Very skilled 



11 . How skilled was the mediator at asking relevant questions? Please check one: 

□ I Not at all skilled 

□ 2 Somewhat skilled 

□ 3 Very skilled 

12 . How skilled was the mediator in helping you to generate and consider options for resolving the dispute? 
Please check one: 

□ I Not at all skilled 

□ 2 Somewhat skilled 
I I 3 Very skilled 



13 . How knowledgeable of relevant laws and regulations was the mediator? Please check one: 

□ I Not at all knowledgeable 

□ 2 Somewhat knowledgeable 

□ 3 Very knowledgeable 

14 . Concerning the impartiaUty of the mediator, how did you feel? Please check one: 

□ 1 The mediator favored the school 

□ 2 The mediator favored the family 

□ 3 The mediator was neutral, and favored neither party 

15. Did the mediator offer specific suggestions for resolution? Please check one: 

□ .No 

□ 2 Yes 



ERIC 



Using Participant Feedback to Evaluate and Improve Quality in Mediation 



27 



The following questions relate to your satisfaction with the mediation process. 

16. How satisfied are you with the outcome of the mediation? Please check one: 

□ 1 Very dissatisfied 

□ 2 Somewhat dissatisfied 

□ 3 Somewhat satisfied 

□ 4 Very satisfied 



17. Concerning the impartiality of the special education mediation program (not the mediator), 
how do you feel? Please check one: 



□ 1 The mediation program favors schools 

□ 2 The mediation program favors famifies 

□ 3 The mediation program is neutral, and favors neither party 



18. Compared to other likely means of resolving this matter, how did mediation affect the cost of this case 
to you or your agency? Please check one: 



□ 1 Mediation cost more 

□ 2 No change in cost 

□ 3 Mediation cost less 

□ 4 Don’t know 



19. Compared to other likely means of resolving this matter, how did mediation affect the time spent 
addressing this matter? Please check one: 



□ 1 Mediation will increase the time spent 

□ 2 No change in time spent 

□ 3 Mediation will shorten the time spent 

□ 4 Don’t know 



20. In case this evaluation has failed to cover them, what actions of the mediator — good or bad — 
were most important to you? 



21. Please provide any suggestions for improvement related to the mediation services you received. 
What could be done differently? 
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Sample follow-up survey — The following survey may be administered as a written questionnaire 
or an interview. 

A few months ago you participated in a mediation regarding special education services. This survey is a 
follow-up to inquire about the impact of the mediation today. Your responses will be confidential. 

Please identify your role in the mediation: 

□ 1 Family member 

□ 2 School representative 

□ 3 Family advocate or attorney 

□ 4 School advocate or attorney 

/. If an agreement was reached in mediation, has it been effective in addressing the issues raised in mediation? 

□ .No 

□ 2 Yes 

2 . If an agreement was reached in mediation, have the provisions of the ^reement been followed or implemented? 

□ 1 None of the provisions have been followed or implemented 

□ 2 Some (but not all) provisions have been followed 

or implemented 

□ 3 All provisions have been followed or implemented 



5 . If an agreement was not reached in mediation, how was the matter resolved? 

□ 1 Through a due process hearing 

□ 2 Through a complaint filing 

□ 3 Through an lEP meeting 

□ 4 Through an informal agreement or settlement 

□ 5 The concern was dropped 

□ 6 The concern remains outstanding 



4 . When you compare your situation before and after the mediation, how has mediation affected your 
relationship with the other participants? Please check one: 

□ 1 Mediation has harmed the relationship 

□ 2 Mediation has had Uttle or no effect on the relationship 

id I I 3 Mediation has improved the relationship 
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5. How productive or unproductive was mediation for this matter? Please check one: 



I I 1 Mediation was very unproductive 
I I 2 Mediation was somewhat unproductive 

□ 3 Mediation was somewhat productive 

□ 4 Mediation was very productive 

6 . Would you recommend mediation to another family or school in a similar dispute? 

□ .No 

I I 2 Yes 

7. Please provide any suggestions for improvement related to the mediation services you received. 
What could be done differently? 
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Sample cover letter for mailed follow-up surveys 



Dear 



A few months ago you participated in a mediation through [SEMS agency] . 

You may have completed a questionnaire at the conclusion of the mediation. The enclosed 
survey is shorter and is designed to help us measure the impact of the mediation over time. 
We ask that you take a few minutes to complete and return the enclosed survey at your 
earhest convenience. 

The [SEMS agency] conducts ongoing evaluation of its services, and your input is 
essential to that evaluation. The feedback you share on the enclosed survey will be treated as 
confidential and anonymous — when you return the survey, there will be no record of your 
identity. The code number on the survey form is a temporary identifier used only for tracking 
purposes — we may send reminders to survey recipients who do not return this form. 

For your convenience, an addressed, postage-paid envelope is included. 

Thank you very much for your assistance with our evaluation. If you have any 
questions about the survey, please contact at . 



Sincerely, 
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Readings and resources on dispute resolution evaluation 

Costantino, C. A., & Merchant, C. S. {\^%) . Designing conflict management systems. 

San Francisco; Jossey-Bass Publishers. 

Dispute Systems Design Working Group. . Performance indicators for ADR program evaluation. 
Administrative Conference of the United States, Washington, D.C. 

Engiles, A., Peter, M., Baxter Quash-Mah, S., &Todis, B. (1996). Team-based conflict resolution in 
special education. Eugene, OR: Lane County Direction Service. 

Engiles, A., Fromme, C., LeResche, D., & Moses, P. (1999) • Roys to access: Encouraging the use of mediation 
by families from diverse backgrounds. Eugene, OR: CADRE. 

(Also available from; http://www.directionservice.org/cadre/keys.cfm) 

Fetterman, D.M., Kaftarian, S., & Wandersman, A. (Eds.). {l^%) . Empowerment evaluation: Knowledge and 
tools for self-assessment and accountability. Newbury Park, CA: Sage Pubhcations. 

McDougall, D. (1998). Evaluating the fiscal effectiveness of ADR processes in special education settings. 
(Available from: http://www.directionservice.org/cadre/articles/evaladr.htm) 

Ostermeyer, M., & Keihtz, S. L. . Monitoring and evaluating court-based dispute resolution programs: 
A guide for judges and court managers. WiUiamsburg, VA: National Center for State Courts. 

Patton, M. Q. (1996). Utilization-focused evaluation. Thousand Oaks, CA: Sage Pubhcations. 

Rolph, E., & MoUer, E. (1995). Evaluating agency alternative dispute resolution programs: A users’ guide 
to data collection and use. Santa Monica, CA: RAND Corporation, Institute for Civil Justice. 

Schrag, J., & Schrag, H. L. . Status of annual dispute resolution effectiveness and impact measurement, 
1998-1999. Alexandria, VA: National Association of State Directors of Special Education. 

Stienstra, D. (2000) . Rules of thumb for designing and administering mailed questionnaires. 

Washington, DC: Federal Judicial Center. 
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CADRE appreciates feedback on its 
products and technical assistance. 

Please answer the following questions 
and return this card so that we can 
continue to improve our services. 

Please check any product(s) 
you are commenting on: 

I I “Using Participant Feedback . . . " 

I I “Beyond Mediation...” 

I I “Families and Schools... ” 

I I “Considerations for Mediating ... ' 

□ 



D Was the enclosed publication of value to you? YES / NO 

□ Do you plan to share the pubhcation with others? YES / NO 

If you have used CADRE services or the CADRE web site: 

Q Did you get the information you needed? YES / NO 

□ Was the information provided within a reasonable amount of time? YES / NO 

□ Were CADRE staff members helpful? YES / NO 

XFhat impact did CADRE'S information/assistance have on a child, 
a family, or a professional? Other comments? 




CADRE appreciates feedback on its 
products and technical assistance. 

Please answer the following questions 
and return this card so that we can 
continue to improve our services. 

Please check any product(s) 
you are commenting on: 

I I “Using Participant Feedback. . . ” 

I I “B^md Mediation...” 

□ “Families and Schools... ” 

□ “Considerations for Mediating ... ' 

□ 



□ Was the enclosed pubhcation of value to you? YES / NO 
B Do you plan to share the pubhcation with others? YES / NO 

If you have used CADRE services or the CADRE web site: 

B Did you get the information you needed? YES / NO 
Q Was the information provided within a reasonable amount of time? YES / NO 
B Were CADRE staff members helpful? YES / NO 

What impact did CADRE'S informatim/assistance have on a child, 
a family, or a professional? Other comments? 
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CADRE appreciates feedback on its 
products and technical assistance. 



Q Was the enclosed pubhcation of value to you? YES / NO 

B Do you plan to share the pubhcation with others? YES / NO 

If you have used CADRE services or the CADRE web site: 

B Did you get the information you needed? YES / NO 
Q Was the information provided within a reasonable amount of time? YES / NO 
B Were CADRE staff members helpful? YES / NO 



Please answer the following questiom 
and return this card so that we can 
continue to improve our services. 

Please check any product (s) 
you are commenting on: 

□ “Using Participant Feedback... ” 
f~~) “Beyond Mediation. . . ” 

n “Families and Schools...” 

O “Considerations for Mediating... ” 

□ 



What impact did CADRE'S information/assistance have on a child, 
a family, or a professional? Other comments? 
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Resotution in Speciai Education 







CADRE is funded by the United States 
Department of Education as the National Center 
on Dispute Resolution and Special Education. 

CADRE leverages the experience of individual 
and organizational leaders to provide efficient, 
effective and high quality technical assistance. 
These affiliations include a unique blend of 
parents and professionals, expertise in technical 
assistance and extensive knowledge of dispute 
resolution practices. 

CADRE builds its activities, products and services 
around the needs of state and local education 
and early intervention systems, parent centers, 
famihes and educators. 

CADRE uses advanced communications 
technology and traditional strategies to: 

□ Provide technical assistance on implementation 
of the mediation requirements under IDEA ‘97. 

□ Motivate parents, schools, and service providers 
to use appropriate dispute resolution processes. 

□ Stimulate and support efforts to resolve disputes 
early and effectively. 
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